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Abstract. Have you ever heard the saying "frustration is written all over your face"? Well this saying is true, but that is not the only place. 
Frustration is written all over your face . . . and your body. The human body has various means to communicate an emotion without utterance of a 
single word. The Media Equation says that people interact with computers as if they are human; this includes experiencing frustration. This 
research measures frustration by monitoring human body-based measures such as heart rate, posture, skin temperature, and respiration. The 
OCC Theory of Emotions is used to separate frustration into different levels or intensities. The results of this study showed that individual 
intensities of frustration exist, so that task performance is not degraded. Results from this study can be used by usability testers to model how 
much frustration is needed before task performance measures start to decrease. 


1.0 Introduction 

Interruptions and task-blocking events seem to 
be common place in human-computer interactions. 
These task inhibitors contribute to user frustration 
and are frequent in most users' interactions with 
computers. Researchers in human-computer 
interaction have studied extensively ways to reduce 
the amount of user frustration experienced by 
humans; however this emotion continues to occur 
and varies according to the individual. Since each 
person is unique, perhaps it is necessary for us to try 
to understand the relationship between user 
frustration and its impact on task performance. In 
this research, the OCC Theory of Emotions is used 
to separate frustration into its varying amounts, 
levels, or intensities to understand the unique 
relationship between each level and user 
productivity measures. 

2.0 Body 

2.1. The Media Equation 

The Media Equation theorizes that humans treat 
new media such as cell phones, tablets, laptops, and 
video games as if they are interacting with real 
persons. This display of emotion includes positive 
and negative emotions. Positive emotions displayed 
after completing a very important task, signal that 
activity toward a goal can end and resources can be 
reallocated for other tasks. However, negative 
emotions produced after being interrupted from 
completing a very important task cause an opposite 
response and motivate users to engage in actions to 
set things right or prevent unpleasant events from 
occurring . User frustration is one such negative 
emotion that can occur in human-computer 
interactions. It is the frustration that occurs when a 
human is interacting with a computer system. It is 
experienced by users of any electronic device; 


however it is most often associated with desktops, 
laptops, tablets, and any other portable media. 

2.2 Affective Computing 

Affective computing is interested in giving 
computers the ability to interpret human emotions 
from human body-based measures such as heart 
rate, skin temperature and respiration measures . 
User frustration has been measured physiological 
indicators such as increased heart rate, skin 
temperature, and respiration. The human body has 
various ways to communicate user frustration. 
Some of these ways include increasing the amount 
of blood pumped to the heart, thereby increasing the 
temperature of the skin, and increasing the amount 
of oxygen taken in by the lungs. 

Most research in human-computer interaction 
and affective computing has focused on ways to 
reduce the amount of user frustration experienced 
by a human . However, examining the relationship 
between the amounts of user frustration and task 
performance measures has not been studied. This 
paper outlines the use of physiological indicators of 
frustration, along with the OCC Theory of emotions 
to model how much user frustration is needed before 
task performance measures start to decrease. The 
relationship between user frustration and task 
performance is also explored in this research. 
Specifically, the OCC Theory of Emotions was used 
to separate frustration into various intensities or 
amounts to understand the individual impact of each 
level of frustration on task performance measures. 

2.3 OCC Theory of Emotions 

The OCC Theory was developed in 1988 by 
Ortony, Clore, and Collins as a cognitive structure to 
understanding emotions and how they are displayed 
. Under this theory, there are 22 emotion types and 
each emotion type has different factors affecting the 
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intensity of an emotion. Factors that tend to increase 
emotion intensity often increase the potential for 
other emotions to occur in that emotion type. 

OCC was created for use in artificial intelligence 
research for emotion synthesis. A key premise in 
this theory, is emotions have not occurred until they 
have reached and surpassed a set threshold, Figure 
2. Also, once emotions have surpassed this 
threshold, they are activated and taken on various 
intensities or amounts as calculated in the original 
OCC Theory . 

Another key premise in OCC Theory is that 
people perceive the world to be events, agents, and 
objects and emotion are tightly coupled together. 
Emotions occur due to consequences of events, 
actions of agents, or aspects of objects. For 
instance, a student may have a deadline to submit 
his/her homework assignment and his/her computer 
crashes. Initially, the student may feel extreme 
anger towards his/her computer. However, as the 
consequences of the event unfold, he/she may 
begin to feel fear over missing the submission 
deadline. In the example given, the computer is the 
agent, the action of the agent is the computer 
crashing, and failing the homework assignment is 
the consequence of the computer crashing. 

3.0 Methods 

The objective of this work is to try and understand 
the relationship between the amount of user 
frustration experienced and task performance. To 
study this, participants were asked to interact with an 
application that included system delays of 500 ms 
while wearing the BioPac Bio Harness. Individuals 
would first experience a control condition that would 
account for the user’s baseline mood and other 
factors that could affect the test. The actual 
experiment part of the study included adding the 
system delays of 500 ms to communications 
between the mouse and the keyboard. 

Measures such as heart rate, skin temperature, 
posture, and breathing rate were collected from each 
individual. Productivity measures were calculated as 
well including: the number of times a subject skipped 
a task, consecutive number of typos, number of 
formatting errors, number of uncompleted task, 
number of times student did not follow directions, 
etc. These measures were taken during the control 
and experiment part of the study and then used 
along with intensities to determine what intensities 
caused task performance to decrease. 


OCC Theory was also adapted for the emotion 
type anger since frustration is not included in the 
original theory. Figure 1, and assumed the control 
study without the added frustrating events provided 
a baseline for the subject’s usual behavior. This was 
also done for task performance measures. 

if (nafuff > control high) 
intensity - value — control high 
else if( value < con ere I Jew) 
ftitentfry = control low- turfue 
else intensity = 0; 

Figure 1. Adaptation of OCC Theory of Emotions. 

To understand the relationship between task 
performance and user frustration, we counted the 
number intensities where task performance metrics 
remain unchanged, the number of intensities where 
task performance decreased, and the number of 
intensities where task performance increased and 
compared these measures for each individual in the 
study. 

44 subjects participated in the study, females and 
23 males, and two were excluded because their task 
performance data was missing or incomplete. Bio- 
signal intensities were calculated for each of the 
subjects in the study and task performance was 
used to compute the total number of intensities that 
decreased, increased, or left task performance 
unchanged. 

IF (EMOTION POTENTIAL) > (EMOTION THRESHOLD) 

THEN 

SET (EMOTION INTENSITY) - (EMOTION POTENTIAL) 
(EMOTION- THRE 5HOLD) 

ELSE 

S ET ( EMOTION ■ I NTENS I TV ) = 0 ; 

Figure 2. Original OOC ComputaiitmaJ Theory of Emotions. 

The lab practitioner conducting the study asked 
informal questions after the study to understand the 
mood of the user upon finishing. Users were asked 
to provide their observations about their level of 
frustration while interacting with the web- page that 
included system delays. During the study, users 
were given the option to skip a task and go on to the 
next tab, if they did not want to complete the task. 

This study is in compliance with the George 
Washington University's Office of Human Subjects 
Internal Review Board. All applicable documentation 
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can be found against the IR8 number 110732. All 
human subjects were provided with an approved 
informed consent form. 


observations taken after the experiment were not 
always congruent with the average intensity 
calculated by the OCC Theory of Emotions. 


4.0 Results 

Results from the study showed that on average 
users were able to complete the task assigned to 
them without going over the allotted time of 15 
minutes. The duration of the study with added 
delays was faster. This was surprising since the 
study included system delays that created a barrier 
to task completion. 

The majority of participants did not skip a task. 
When asked why participants did not skip a task and 
go on to the next, some subjects said they wanted to 
follow directions and felt that the task was easy 
enough without skipping a task. A few users opted to 
skip a task, but then went on to complete the 
skipped task before the end of the study. The only 
participant to skip a task and stay with his decision 
was the oldest participant, 45 years of age, in the 
study. 
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Results from the study showed that users' 
observations and lab practitioners frustration 


5.0 Conclusion and Discussion 

Users are more familiar with frustrating incidents 
than ever before due to the ubiquity of computers. 
Although developers, testers, and usability 
engineers have improved the computing interfaces, 
frustrations still linger. With the everyday use of cell 
phones, TVs, tablets, mp3 players, etc., users have 
al, but become accustomed to certain frustrations. 
Although frustration occurs as shown in the graphs 
above, it seems that the normal user, as observed in 
this study, has adapted themselves to the events. 
Perhaps , it is necessary to understand and interpret 
the frustration events and limit their frequency rather 
than adapting the interface. 

From analysis of the graphs above, there seems 
to exist intensities that allow a user to be productive 
without affecting task performance. These 
intensities can better assist usability testers and 
developers if one can maximize the occurrence of 
such intensities. 

6.0 Future Work 

Future work from this research includes using 
bio-signal data, task performance measures, and 
contextual data about what artifacts or widgets a 
person is interacting with while using an application. 
Perhaps including contextual information, along with 
bio-signal and task performance measures could 
allow developers to gain more insight into what 
widgets users are encountering the most problems. 
Also, collecting localized task performance 
measures along with bio-signal measures taken at 
longer intervals will help relate these two measures 
together. These measures, if used during usability 
testing, could provide a more useful metric in 
determining how a person really feels about a new 
application. 

An example of this would be in human-in-the-loop 
usability testing by providing a new user with a 
system that they have never worked with before. 
Contextual information such as what widget a 
person clicked is recorded, along with the number of 
discrete measurable tasks completed and the time in 
between each task completed, along with on the fly 
calculations of user frustration intensities. Since 
each user is different, perhaps if a certain population 
of the users are experiencing frustration levels 
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above a certain threshold set by usability engineers, 
developers could leverage this information to 
analyze parts of the user interface that is causing the 
decrease in productivity measures. 
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